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Abstract 


All  shape-from-intensity  methods  assume  that  points  in  a  scene  are  only  illuminated  by  sources  of 
light.  Most  scenes  consist  of  concave  surfaces  and/or  concavities  that  result  from  multiple  objects 
in  the  scene.  In  such  cases,  points  in  the  scene  reflect  light  between  themselves.  In  the  presence 
of  these  interreflections,  shape-from-intensity  methods  produce  erroneous  (pseudo)  estimates  of 
shape  and  reflectance.  The  pseudo  shape  and  reflectance  estimates,  however,  are  shown  to  carry 
information  about  the  actual  shape  and  reflectance  of  the  surface.  An  iterative  algorithm  is  presented 
that  simultaneously  recovers  the  actual  shape  and  the  actual  reflectance  from  the  pseudo  estimates. 
The  recovery  algorithm  works  on  Lambertian  surfaces  of  arbitrary  shape  with  possibly  varying 
and  unknown  reflectance.  The  general  behavior  of  the  algorithm  and  its  convergence  properties 
are  discussed.  Both  simulation  as  well  as  experimental  results  are  included  to  demonstrate  the 
accuracy  and  stability  of  the  algorithm. 
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1  The  Interreflection  Problem 


We  address  a  challenging  vision  problem  that  has  remained  unsolved  for  the  past  two  decades. 
Surface  elements  in  a  scene,  when  illuminated,  reflect  light  not  only  in  the  direction  of  the  sensor 
but  also  between  themseives.  This  is  always  the  case  except  when  the  scene  consists  of  only  a  single 
convex  surface.  These  interreflections,  also  referred  to  as  mutual  illuminations,  can  appreciably 
alter  the  appearance  of  the  scene.  None  of  the  existing  vision  algorithms  reason  about,  or  even  take 
into  account,  the  effects  of  interreflections.  Consequently,  interreflections  often  confuse  vision 
algorithms  and  cause  them  to  produce  erroneous  results. 

A  class  of  vision  algorithms  that  are  particularly  affected  by  interreflections  are  shape-from- 
intensity  algorithms,  such  as,  shape-from-shading  [7],  photometric  stereo  [14],  and  photometric 
sampling  [12].  All  these  methods,  are  based  on  the  assumption  that  points  in  the  scene  are 
illuminated  only  by  the  sources  of  light  and  not  other  points  in  the  scene;  interreflections  are 
assumed  not  to  exist.  As  a  result,  existing  shape-from-intensity  methods  produce  erroneous  results 
when  applied  to  concave  surfaces  and  concavities  that  result  from  multiple  objects  in  the  scene. 
As  an  example.  Figure  la  shows  a  concave  Lambertian  surface  of  constant  reflectance  (albedo  = 
0.75),  and  Figure  lb  shows  its  shape  extracted  using  photometric  stereo.  The  inability  to  deal  with 
interreflections  has  in  the  past  limited  the  utility  of  shape-from-intensity  methods. 


(b) 


Figure  1:  (a)  A  concave  surface,  (b)  Its  shape  extracted  using  photometric  stereo. 


We  identify  two  separate  problems  associated  with  interreflections;  the  forward  (graphics) 
problem  and  the  inverse  (vision)  problem.  All  previous  work  done  in  this  area  is  related  to  the 
forward  problem.  The  forward  problem,  involves  the  prediction  of  image  brightness  values  given 
the  shape  and  reflectance  of  a  scene.  Hom  [5]  discussed  the  changes  in  image  intensities  due  to 
interreflections  caused  by  polyhedral  surfaces  that  are  Lambertian  in  reflectance.  Koenderink  and 
van  Doom  [9]  formalized  the  interreflection  process  for  Lambertian  surfaces  of  arbitrary  shape 
and  varying  reflectance  (albedo).  They  proposed  a  solution  to  the  forward  problem  in  terms  of 
the  eigenfunctions  of  the  interreflection  kernel.  Cohen  and  Greenberg  [1]  modeled  the  scene  as 
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a  finite  collection  of  Lambertian  planar  facets  and  proposed  a  radiosity  solution  to  the  forward 
problem  and  used  it  to  render  images  for  graphics.  Recently,  Forsyth  and  Zisserman  [3]  used  a 
similar  numerical  solution  to  the  forward  problem  to  compare  predicted  image  intensities  with 
experimentally  obtained  image  intensities  for  a  set  of  Lambertian  surfaces  of  known  shape  and 
reflectance. 

Our  goal  here  is  to  solve  the  inverse  (vision)  problem.  Given  image  intensities,  we  wish 
to  recover  the  shape  and  reflectance  of  the  scene  in  the  presence  of  interreflections.  The  inverse 
interreflection  problem  is  a  particularly  difficult  one,  for  in  its  ambiguity,  it  resembles  the  well- 
known  "chicken  and  egg  problem"  (Figure  2).  If  we  can  model  the  interreflection  effects,  we  may  be 
able  to  compensate  scene  images  for  these  effects  and  extract  accurate  shape  information.  However, 
it  is  obvious  that  modeling  interreflections  requires  prior  knowledge  of  shape  and  reflectance.  But 
it  is  shape  that  we  are  attempting  to  recover!  So  which  one  comes  first,  shape  or  interreflections? 


Figure  2:  The  inter-dependence  between  shape  and  interreflections  makes  shape  recovery  in 
the  presence  of  interreflections  a  difficult  problem. 

In  this  paper,  we  present  a  solution  to  the  inverse  problem  for  Lambertian  surfaces  of 
arbitrary  (but  continuous)  shape,  with  possibly  varying  but  unknown  reflectance  (albedo).  The 
shape  and  reflectance  recovery  algorithm  works  as  follows.  First,  a  local  shape-from-intensity 
method  is  applied  to  the  concave  surface  to  obtain  "pseudo"  (erroneous)  estimates  of  shape  and 
reflectance.  Our  solution  is  based  on  the  observation  that  the  pseudo  shape  and  reflectance,  though 
erroneous,  carry  information  about  the  actual  shape  and  reflectance  of  the  surface.  The  pseudo 
shape  and  reflectance  are  used  to  model  the  interreflection  effects.  We  show  that  the  pseudo 
shape  is  "shallower"  than  the  actual  shape  and  hence  exhibits  weaker  interreflections.  These 
interreflections  are  used  to  compensate  the  pseudo  shape  and  reflectance  estimates  to  obtain  a 
better  (more  concave)  shape  estimate  and  reflectance  information.  This  shape  and  reflectance  is 
again  used  to  model  interreflections  to  obtain  even  more  accurate  shape  and  reflectance  estimates 
from  the  pseudo  estimates.  In  this  manner,  shape  and  reflectance  estimates  are  iteratively  refined  to 
finally  converge  to  the  correct  shape  and  reflectance.  A  detailed  analysis  of  convergence  is  given  for 
the  simple  case  of  two  planar  surface  elements.  Convergence  for  the  more  general  case  is  discussed 
and  demonstrated  by  numerous  simulation  results.  Several  experimental  results  are  included  to 
demonstrate  the  robustness,  accuracy,  and  practical  feasibility  of  the  proposed  algorithm. 
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2  Modeling  Tnterreflections 


Our  solution  to  the  inverse  interreflection  problem  is  based  on  the  solution  to  the  forward  problem; 
modeling  interreflections  for  a  surface  of  given  shape  and  reflectance.  Hence,  this  section  will 
serve  as  background  theory  for  subsequent  sections.  The  interreflection  model  that  we  describe 
here  is  primarily  based  on  the  formulation  proposed  by  Koenderink  and  van  Doom  [9].  All  surfaces 
in  the  scene  are  assumed  to  be  Lambertian.  We  will  shortly  see  that  this  assumption  is  necessary 
to  obtain  a  closed  form  solution  to  the  forward  interreflection  problem.  The  Lambertian  surface 
can  have  any  arbitrary  shape  and  varying  reflectance,  i.e.  albedo  value  ( p )  may  vary  from  surface 
point  to  surface  point.  In  deriving  the  interreflection  model,  we  will  use  radiometric  concepts  such 
as  irradiance  and  radiance  which  are  defined  in  Appendix  A.l. 


2.1  Analytic  Forward  Solution 


Consider  the  concave  surface  \(u,  v)  shown  in  Figure  3a.  We  are  interested  in  finding  the  radiance 
L(x)  of  the  point  x  due  to  the  radiance  L(x')  of  the  point  x'.  The  point  x  can  be  illuminated  by  the 
point  x'  only  if  the  two  points  can  "see"  each  other.  The  visibility  or  View  function  is  defined  as: 


View  (x ,  x') 


n  .(-!•)+  |  n.(-r)  |  n' .  r  +  j  n' .  r  | 
2  |  n . (-r)  j  2  |  n'.r  | 


(1) 


where  n  and  n'  are  unit  surface  normal  vectors  at  the  points  x  and  x\  respectively,  and  r  is  the 
vector  from  x  to  x'.  The  function  View(\,  x')  equals  unity  when  the  two  points  can  illuminate  each 
other  and  zero  otherwise.  The  radiance  of  the  point  x  is  related  to  its  irradiance  as: 


Figure  3:  (a)  A  concave  surface  in  three-dimensional  space,  (b)  Two  surface  elements  that 
are  visible  to  one  another. 
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(2) 


L(x)  =  ^£(x) 

7T 

where  p(x)  is  the  albedo  function,  and  the  factor  p(x)/ it  is  the  bi-directional  reflectance  distribution 
function  (Appendix  A.l)  for  a  Lambertian  surface.  The  irradiance  £(x)  of  the  surface  element  dx 
due  to  the  radiance  of  the  point  x'  may  be  derived  using  the  geometry  shown  in  Figure  3b: 


E(x)  = 


ln.(-r)].[n'  rj.rfx'  View(x,  x') 

II  r  IP 


L(x') 


From  equations  2  and  3,  we  obtain: 

.  ,  ,  P  (x)  [  [  n .  (-r)] .  [  n' .  r] .  dx' .  View  (x.  x') 
L  (x)  =  -  - rnr rr, - 


L(x') 


From  a  geometrical  perspective,  the  interreflections  between  points  x  and  x'  are  governed  by  the 
factor: 

i\  [n.(-r)].[n'.r].t/x'. View(x.  x') 

K(x,x')  =  ^ - jjTjp - j  (5) 

K(x,  x')  is  called  the  interreflection  kernel  and  is  a  symmetric,  positive  definite  function.  It 
vanishes  for  surfaces  points  that  do  not  illuminate  one  another  either  due  to  their  orientations  or  due 
to  occlusion  by  other  points.  The  kernel  is  bounded  by  the  geometrical  constraint  that  no  surface 
element  can  radiate  in,  or  receive  radiations  from,  more  than  a  half-space  around  it.  Hence: 


1 1  \\  dxdx'  <  l 


Now  assume  that  the  concave  surface  in  Figure  3a  is  illuminated  by  a  single  distant  point 
source  of  light  in  the  direction  s.  Then,  the  radiance  of  the  surface  due  to  the  source  alone  (excluding 
interreflection  effects)  may  be  expressed  in  terms  of  the  irradiance  of  the  surface  by  the  source: 

Ls(x)  =  ^£,(x)  (7) 

rr 

The  irradiance  due  to  the  source  is  the  flux  incident  per  unit  area  of  the  surface  and  for  a  Lambertian 
surface  is  proportional  to  the  cosine  of  the  angle  between  the  source  direction  and  the  surface  normal 
direction,  i.e.  £,(x)  =  kn.s.  The  constant  of  proportionality  k  is  determined  by  the  radiant  intensity 
of  the  source  and  its  distance  from  the  surface. 

The  total  radiance  of  x  is  then  expressed  as  a  sum  to  the  radiance  due  to  the  source  and  the 
radiance  due  to  all  other  points  on  the  surface: 

£(x)  =  Ls(x)  +  —  /  K(x,x')L(x')dx'  (8) 

7 r  J 

The  above  equation  is  referred  to  as  the  interreflection  equation.  It  is  similar  in  form  to  the 
Fredholm’s  integral  [9]  f5]  and  does  not  lend  itself  to  a  straightforward  solution.  However,  if  all 
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poinr'  jn  the  concave  surface  have  the  same  reflectance  (p(\)  =  p),  a  solution  to  L(x)  (the  forward 
interreflection  problem)  is  given  by  the  Neumann  series  as: 

L(\)  =  LA' 

where 

Km  (x,  x')  = 
and  K  i  = 

The  following  observations  are  made 

•  It  is  important  to  note  that  the  above  solution  is  valid  only  under  the  Lambertian  assumption. 
For  Lambertian  reflectance,  the  radiance  of  a  surface  point  is  independent  of  the  vantage 
point.  As  a  result,  both  L  (x)  and  L  (r/)  are  constants  in  equation  8  and  hence  a  solution  can 
be  obtained. 

•  The  solution  is  iterative  in  nature;  it  is  an  infinite  sum  of  the  kernels  Km  that  must  each  be 
evaluated  using  the  previous  kernel  Km-i. 

•  The  solution  may  be  interpreted  as  a  mathematical  representation  of  the  "ray-tracing"  process 
that  is  often  used  in  the  area  of  computer  graphics.  The  m,h  iteration  explicitly  represents  the 
contribution  of  the  m  times  interreflected  rays. 

•  Though  the  Neumann  series  is  an  infinite  one,  the  solution  is  guaranteed  to  converge  to  a 
finite  value.  This  is  because  p{\)  <  1  for  all  surface  points,  and  hence,  the  series  diminishes 
to  zero  as  m  approaches  infinity.  This  is  consistent  with  our  real-world  experience;  diffuse 
concave  surfaces  that  exhibit  interreflections  never  appear  to  be  infinitely  bright. 


0  +  £  Pm  /  Km(\.  x')L(x')dx'  (9) 

m= i  J 

J  ^*iylKm_j{y,x')dy  (m  >  2) 

K 

z 

with  respect  to  the  above  solution: 


2.2  Numerical  Forward  Solution 


Discretization  of  the  concave  surface  leads  to  a  more  elegant  forward  solution  than  the  Neumann 
series.  The  following  solution  has  been  previously  used  to  render  discrete  images  in  graphics  [1] 
and  to  compare  experimentally  obtained  image  intensities  with  predicted  intensities  [3].  Let  us 
assume  the  surface  to  be  comprised  of  m  facets  as  shown  in  Figure  4.  The  radiance  and  albedo 
values  of  each  facet  i  are  assumed  to  be  constant  over  the  entire  facet  and  equal  to  the  radiance  and 
albedo  values  at  the  center  point  x,  of  the  facet,  i.e.  L,  =  L(x,)  and  pi  =  p(x,).  Then  we  can  write 
equation  8  as: 


L, 


LSI  + 


K  (x„  Xj)  dXj 


(10) 
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where  Sj  is  the  entire  surface  of  the  facet  j.  We  can  define  the  discrete  form  of  the  interreflection 
kernel  as: 

Kij  =  Js  K  (x„  \j)dXj  (11) 

where  Ktl  is  undefined  and  Kt]  vanishes  for  facet  pairs  that  are  not  visible  to  one  another.  The 
discrete  form  of  the  interreflection  equation  can  therefore  be  written  as: 

Li  =  Lsi  +  —  '^LjKij  (12) 

7T  .  . 

J# 


T 

We  define  the  facet  radiance  vector  as  L  =  [  Lj,  L2, . ,  Lm  ]  and  the  source  contribution  vector 

T 

as  Ls  =  [  Lsl,  Ls2, . .  Lsm  Y  .  We  also  define  the  albedo  matrix  P  and  the  kernel  matrix  K  as: 


pi  0  .  . .  0 

OK  12 .  1 

0  p20  .  .  0 

K21  0 . 

K  = 

. 0 . 

. 0 

0  0  ...  pm 

. 0 

Then,  equation  12  may  be  written  as: 


(13) 


L  =  Ls  +  PKL 


(14) 


Figure  4:  Modeling  the  surface  as  a  collection  of  facets,  each  with  its  own  radiance  and 
albedo  values. 
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or: 


(I  -  PK)L  =  Ls 


(15) 


where  I  is  the  identity  matrix.  Hence,  we  find  that  discretization  of  the  surface  enables  us  to  obtain 
a  non-iterative,  closed-form  solution  to  the  forward  interrefi.ection  problem.  The  kernel  and  albedo 
matrices  are  determined  by  the  shape  and  reflectance  of  the  surface,  respectively.  The  source 
direction  and  intensity  may  be  used  to  obtain  the  source  contribution  vector  Ls.  Then  the  radiance 
of  the  surface  facets,  L,  may  be  determined  using  the  above  equation. 

Equation  14  explicitly  describes  the  radiance  of  a  facet  as  the  sum  of  its  radiance  due  to 
the  sourct  nd  the  contributions  of  other  facets.  Loosely  speaking,  this  may  be  interpreted  as  a 
weighted  averaging  of  radiance  values  in  the  direction  of  concave  curvature  that  tends  to  subdue 
the  visual  conspicuousness  of  surface  concavity. 

We  would  like  to  conclude  this  section  with  a  brief  note  on  the  size  of  individual  facets. 
We  have  assumed  that  the  radiance  and  albedo  are  constant  over  the  facet  area.  This  assumption 
is  valid  only  when  the  facets  are  planar  and  infinitesimally  small.  While  solving  the  forward 
interreflection  problem,  we  are  free  to  select  appropriate  (small)  facet  sizes.  In  solving  the  vision 
problem,  however,  we  are  limited  by  the  resolution  of  the  sensor  used  to  image  the  scene.  The 
image  brightness  at  a  "pixel"  location  is  assumed  to  be  constant  over  the  entire  surface  facet  that 
the  pixel  represents.  From  Figure  5,  we  see  that  the  area  dXj  of  the  facet  may  be  related  to  the  area 
dAj  of  the  pixel  as: 


Figure  5:  The  facet  size  is  determined  by  the  size  of  the  sensor  element  (pixel)  and  the  tilt 
of  the  surface  with  respect  to  the  viewing  direction  of  the  sensor. 
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(16) 


dXj  = 


dAj 
v, .  n, 


where  n;  and  v;  are  the  normal  vector  and  viewing  direction  vector,  respectively,  for  the  facet  j. 
For  the  case  of  orthographic  image  projection,  the  viewing  vector  is  constant  over  the  entire  field 
of  view,  i.e.  v;  =  v.  Using  equations  5,  11,  and  16,  we  determine  the  discrete  kernel  to  be: 


Kij  = 


[ n, . r,j]  ■  [ n,. rJt]  dA} 


Ik/,  IP 


v.n, 


View;, 


(17) 


The  kernel  provides  a  good  approximation  only  when  the  facets  i  and  j  are  both  "small"  and  distant 
from  each  other.  Note  the  size  of  a  facet  also  depends  on  its  tilt  with  respect  to  the  viewing  direction. 
Therefore,  while  using  the  discrete  kernel  we  assume  that  facets  are  not  viewed  at  angles  close  to 
the  grazing  angle.  The  above  kernel  represents  the  interreflections  between  two  facets  positioned 
and  oriented  in  three-dimensional  space.  In  Appendix  A.2,  we  have  also  included  the  kernel  form 
for  the  special  case  of  three-dimensional  surfaces  that  have  single  translational  symmetry. 


3  The  Extracted  "Pseudo"  Shape 

The  previous  section  showed  that  surface  radiance  values  are  affected  by  the  presence  of  interreflec¬ 
tion  effects.  This  indicates  that  if  a  shape-from-intensity  method  is  applied  to  a  concave  surface  it  is 
expected  to  produce  erroneous  estimates  of  shape.  In  order  to  generalize  the  inverse  interreflection 
problem  that  we  are  attempting  to  solve,  we  assume  that  the  reflectance  of  the  Lambertian  surface 
is  also  unknown  and  may  vary  from  point  to  point.  Therefore,  by  the  term  shape-from-intensity, 
we  mean  those  methods  that  extract  both  shape  (orientation)  and  reflectance  (albedo)  information. 
Photometric  stereo  [14]  and  photometric  sampling  [12]  are  examples  of  such  shape-from-intensity 
methods.  In  the  presence  of  interreflections,  these  shape-from-intensities  methods  produce  erro¬ 
neous  shape  as  well  as  erroneous  reflectance  information.  We  refer  to  the  extracted  shape  as  the 
"pseudo  shape"  and  the  extracted  reflectance  as  the  "pseudo  reflectance"  of  the  surface.  In  this 
section,  we  investigate  how  the  pseudo  shape  and  reflectance  are  related  to  the  actual  shape  and 
reflectance  of  the  surface. 

Once  again,  consider  the  surface  comprised  of  m  facets  (Figure  4).  Each  facet  i  is  defined 
as: 

N,  =  -n,  (18) 

7 r 

where  n,  and  p,  are  the  unit  surface  normal  vector  and  the  albedo  for  the  facet  /,  respectively. 
Therefore,  the  term  "facet"  represents  both  local  shape  and  local  reflectance  information.  The 

complete  surface  is  then  defined  by  iht  facet  matrix  F= [  N;,  N2, Nm  ]  .  From  the  interreflection 

equation  (equation  15)  we  can  express  the  surface  radiance  as: 

L  =  (I  -  PKr’L,  (19) 
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Since  the  surface  is  Lambertian,  the  source  contribution  vector  Ls  may  be  determined  from  the 
facet  matrix  F  and  the  source  direction  vector  s  =  [s,,  sy,  sz]r  as: 

L  =  (I  -  PKr'F.s  (20) 

Now  let  us  examine  the  result  of  applying  photometric  stereo  to  the  surface.  Three  source  directions, 
Sj,  s2,  and  Sj,  are  used  sequentially  to  illuminate  the  surface.  We  assume  that  all  three  sources  are 
visible  to  all  facets  on  the  surface.  The  three  resulting  surface  radiance  vectors  LJy  L2,  and  L j  may 
be  expressed  as: 

[L7,L2,L2J  =  (I  —  PK)-1  F.  [S7  ,  S2  ,  Sj]  (21) 

Note  that  the  kernel  matrix  K  and  the  albedo  matrix  P  are  both  invariant  to  the  source  directions 
used  to  illuminate  the  surface.  The  extracted  shape  and  reflectance  information  is  represented  by 
the  pseudo  facet  matrix  Fp  and  is  computed  as: 

Fp  =  [ L;  1  L2  ,  L?  ] .  [s7 ,  s2  ,  s2  ]_1  (22) 

From  equations  21  and  22  we  find  that: 

Fp  =  (I-PKT’F  (23) 

The  i'h  pseudo  facet 1  in  Fp  may  be  written  as: 

N p,-  =  ^nPi  (24) 

where  nPi  and  pPi  are  the  pseudo  surface  normal  and  the  pseudo  albedo  for  the  facet  i  and,  in  the 
presence  of  interreflections,  differ  from  the  actual  surface  normal  and  actual  albedo  of  the  facet. 
We  make  a  few  important  observations  regarding  the  pseudo  facets: 

•  From  equations  23  and  20,  we  see  that  the  pseudo  facets  are  also  Lambertian  in  their 
reflectance!  This  also  implies  that  the  extracted  pseudo  shape  and  reflectance  are  independent 
of  the  source  directions  used  by  the  shape-from-intensity  method  to  illuminate  the  object. 

•  While  the  actual  albedo  values  must  satisfy  the  physical  constraint  p,<  1,  the  pseudo  albedo 
values  tend  to  be  greater  than  the  actual  values  and  for  actual  albedo  values  close  to  unity, 
the  pseudo  albedoes  may  even  exceed  unity  (see  experimental  results  in  section  6). 

•  The  pseudo  orientations  may  be  described  as  a  result  of  the  weighted  averaging  of  actual 
orientations  in  the  direction  of  concave  curvature.  Qualitatively  speaking,  for  concave 
surfaces  the  pseudo  shape  may  be  viewed  as  a  smoothed  version  of  the  actual  shape  and 
appears  to  be  "shallower"  than  the  actual  shape  (see  Figure  1). 


'These  pseudo  facets  are  different  from  the  ones  defined  by  Koenderink  and  van  Doom  in  [9] 
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4  Recovering  Actual  Shape  and  Reflectance 


Our  objective  is  to  simultaneously  recover  both  actual  shape  and  actual  reflectance  of  the  surface 
from  the  extracted  pseudo  shape  and  reflectance.  The  method  is  based  on  the  observation  that 
the  pseudo  shape  and  reflectance  carry  information  about  the  actual  shape  and  reflectance.  A 
closed-form  solution  for  the  actual  shape  and  reflectance,  however,  does  not  seem  possible  as  the 
pseudo  shape  and  reflectance  are  highly  non-linear  functions  of  the  actual  ones.  Hence,  we  seek 
an  iterative  approach. 


4.1  The  Recovery  Algorithm 

Figure  6  illustrates  the  flow  of  the  algorithm.  At  first,  a  local  shape-from-intensity  method  is 
applied  to  the  scene.  If  the  scene  consists  of  a  single  convex  surface,  the  extracted  pseudo  shape 
and  reflectance  are  simply  the  actual  ones.  However,  if  the  scene  consists  of  concavities  resulting 
from  concave  surfaces  and/or  multiple  objects  in  the  scene,  the  pseudo  shape  and 


Figure  6:  The  shape  and  reflectance  recovery  algorithm. 
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reflectance  differ  from  the  actual  ones.  As  we  described  in  the  previous  section,  the  pseudo 
shape  is  expected  to  be  shallower  (less  concave  but  yet  concave)  version  of  the  actual  shape. 
Hence,  the  algorithm  uses  the  pseudo  shape  and  reflectance  as  initial  guesses  of  the  actual  shape 
and  reflectance,  to  model  the  interreflections  and  produce  estimates  for  the  albedo  matrix  P  and  the 
kernel  matrix  K.  It  is  important  to  note  that  the  pseudo  shape  and  reflectance  serve  as  conservative 
initial  estimates,  in  that,  they  produce  interreflections  that  are  greater  than  zero  but  less  than  in  the 
case  of  the  actual  shape  and  reflectance.  The  estimated  P,  K,  and  the  pseudo  facets  Fp  are  then 
inserted  in  equation  23  to  obtain  the  next  estimate  of  the  actual  facets.  This  estimate  of  the  surface 
is  expected  to  be  more  concave  than  the  previous  estimate  and  is  used  in  the  next  iteration  to  obtain 
an  even  "better"  estimate.  The  algorithm  may  be  written  as: 

Fk+J  =  (I  -  P* K* ) Fp  (25) 

where  F°  =  Fp 

In  the  above  equation,  P*  =  P  (F*)  and  K*  =  K  (F*).  Note  that  each  set  of  estimates  of  the 
surface  facets  provides  estimates  of  both  shape  and  reflectance.  With  each  iteration,  more  accurate 
estimates  of  shape  and  reflectance  are  obtained  and  the  result  finally  converge  at  the  actual  shape 
and  reflectance  estimates.  The  convergence  properties  of  the  algorithm  will  be  discussed  later.  We 
now  state  a  few  assumptions  and  observations  related  to  the  above  algorithm. 

•  The  surface  is  assumed  to  be  continuous.  Note  that  the  interreflection  kernel  depends  not 
only  on  the  orientations  of  individual  facets  but  also  their  relative  positions.  Therefore, 
a  depth  map  of  the  scene  must  be  reconstructed  (by  integration)  from  the  orientation  map 
computed  in  each  iteration  of  the  algorithm.  The  continuity  assumption  is  necessary  to 
ensure  integrability  of  the  orientation  maps.  It  appears  that  discontinuities  in  the  depth  of 
scene  points  can  also  be  handled  if  this  information  is  provided  by  a  depth  measurement 
method,  such  as,  stereo. 

•  All  facets  that  contribute  to  the  interreflections  in  the  scene  must  be  visible  to  the  sensor.  It 
is  easy  to  see  that  if  invisible  points  affect  the  radiance  values  of  the  visible  points,  the  kernel 
matrix  would,  in  a  sense,  be  incomplete.  In  such  cases,  the  result  produced  by  the  algorithm 
is  difficult  to  predict  but  would  be  close  to  the  desired  result  if  the  invisible  facets  do  not 
contribute  substantially  to  the  radiance  of  other  facets. 

•  The  proposed  recovery  algorithm  may  be  used  in  conjunction  with  any  local  shape-from- 
intensity  method.  The  shape-from-intensity  method  used  must  be  capable  of  computing 
accurate  estimates  of  both  pseudo  shape  and  pseudo  reflectance2.  The  recovery  algorithm  is 
in  no  way  related  to  the  shape-from-intensity  method  used  to  obtain  the  pseudo  shape  and 
reflectance.  This  fact  is  emphasized  by  the  dotted  line  shown  in  Figure  6. 

2We  do  not  include  shape-from-shading  algorithms  in  this  category  as  they  assume  that  the  surface  has  constant 
albedo  and  that  this  albedo  value  is  known  a-priori. 
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•  No  extra  images  (measurements),  in  addition  to  the  images  used  by  the  shape-from-intensity 
method,  are  needed  to  recover  actual  shape  and  reflectance. 

•  For  each  iteration  of  the  above  algorithm,  the  kernel  is  computed  for  every  pair  of  facets  in 
the  scene.  Therefore,  the  algorithm  is  of  0(M  n2)  complexity,  where  n  is  the  number  of  facets 
in  the  scene  and  M  is  the  number  of  iterations  required  for  shape  and  reflectance  estimates 
to  converge. 


4.2  Simulation  Results 

Figure  7  shows  simulation  results  for  three-dimensional  surfaces  that  are  assumed  to  have  single 
translation  symmetry.  The  form  of  the  interreflection  kernel  for  this  case  is  given  in  Appendix  A.2. 
These  simulation  results  are  included  to  give  the  reader  a  feel  for  the  behavior  of  the  algorithm. 
Experimental  results  for  the  general  three-dimensional  case  as  well  as  the  translation  symmetry  case 
will  be  presented  in  a  later  section.  For  each  surface  in  Figure  7,  the  numerical  forward  solution 
(section  2.2)  was  used  to  predict  the  radiance  of  the  surface  from  its  actual  shape  and  reflectance. 
Facet  radiance  values  for  two  different  source  directions  were  computed  and  a  photometric  stereo 
algorithm  was  used  to  compute  the  pseudo  shape  and  reflectance  estimates.  Note  that  the  single 
symmetry  assumption  reduces  the  problem  to  a  two-dimensional  one  and  only  two  source  directions 
are  necessary  to  compute  facet  orientations  and  albedo  values.  The  pseudo  shape  and  reflectance 
are  then  used  by  the  recovery  algorithm  (equation  25)  to  iteratively  refine  the  pseudo  shape  and 
reflectance  and  finally  converge  at  the  actual  shape  and  reflectance. 

For  the  surfaces  in  Figures  7a  and  7b,  a  constant  albedo  value  of  0.75  over  the  entire  surface 
was  used  to  determine  the  pseudo  shape  and  reflectance.  For  surface  shown  in  Figure  7c,  a  ramp 
function  that  varies  from  0.25  to  0.95  was  used  to  compute  the  pseudo  shape  and  reflectance. 
For  the  surface  shown  in  Figure  7d,  a  checker-board  albedo  function  that  varies  between  0.3  and 
0.7  was  used  to  compute  the  pseudo  shape  and  reflectance.  In  Figure  7d,  some  sections  of  the 
surface  are  occluded  from  other  sections.  While  computing  the  interreflections,  the  algorithm  uses 
geometrical  reasoning  to  determine  if  two  facets  on  the  surface  are  occluded  from  each  other  by 
other  facets.  In  all  of  the  above  cases,  the  recovery  algorithm  did  not  rely  on  prior  knowledge  of 
surface  reflectance,  but  rather  used  the  pseudo  reflectance  along  with  the  pseudo  shape  to  recover 
actual  shape  and  actual  reflectance  simultaneously. 

In  Figures  7c  and  7d  the  pseudo  reflectances  and  the  intermediate  estimates  of  reflectance 
are  also  shown.  For  these  surfaces,  a  convergence  graph  is  also  included  that  shows  the  mean 
orientation  error  9e,  computed  using  all  facets,  for  each  iteration  of  the  recovery  algorithm.  The 
orientation  error  at  a  surface  point  is  defined  as  the  angle  (in  degrees)  between  the  actual  normal 
vector  and  the  estimated  normal  vector.  For  all  of  the  above  surfaces,  and  numerous  other  unreported 
simulation  results,  the  algorithm  converges  smoothly  to  zero  error  in  both  shape  and  reflectance. 
Fairly  accurate  shape  estimates  are  usually  obtained  in  about  7  iterations  of  the  algorithm. 
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Figure  7:  Simulation  Results:  For  each  surface,  the  pseudo  shape  and  pseudo  reflectance 
are  computed  from  the  actual  shape  and  actual  reflectance  using  the  forward  solution  (section 
2.2).  The  recovery  algorithm  is  applied  to  the  pseudo  shape  and  reflectance  to  recover  the 
actual  shape  and  reflectance. 
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5  Convergence 

In  this  section  we  study  the  convergence  properties  of  the  shape  and  reflectance  recovery  algorithm. 
Since  the  interreflection  equation  is  a  highly  nonlinear  function  of  the  shape  and  reflectance  of  the 
surface,  it  is  difficult  the  prove  the  convergence  of  the  algorithm  for  general  case  of  arbitrary  shape 
and  reflectance.  Therefore,  we  start  by  analyzing  the  convergence  properties  for  the  simplest  case 
of  two  planar  facets  and  later  extend  our  analysis  to  the  general  case. 

5.1  Two-Facet  Case 

Consider  two  infinitesimal  planar  facets  of  equal  size  that  are  separated  by  a  distance  r  (Figure  8a). 
We  identify  an  axis  (dotted  line)  with  respect  to  which  the  two  facets  are  symmetrically  oriented 
and  positioned.  The  unit  normal  vectors  n ;  and  n2  are  coplanar  and  therefore  are  defined  by  just  two 
parameters,  namely,  ny  and  nz.  As  a  result  of  the  symmetrical  facet  arrangement,  the  interreflection 
kernels  for  the  two  facets  are  equal,  i.e.  Kj2  -  K21  =  K.  Further,  we  assume  that  the  two  facets 
have  equal  albedo  values,  i.e.  pj  =  p2  =  p.  By  applying  a  shape-from-intensity  method  to  the  two 
facets,  their  pseudo  facets  can  be  computed.  From  equation  23  we  see  that  the  actual  facets  may 
be  expressed  in  terms  of  the  pseudo  facets  as: 


(a) 


(b) 


Figure  8:  (a)  The  two-facet  case,  (b)  The  line  constraint.  All  intermediate  estimates  of  the 
facet  N/  must  lie  on  the  line  passing  through  the  vector  C. 
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(26) 


N,  =  NP]  -  PK NP2 

N2  =  N„  -  pKNP1 

where  N j  and  N2  are  the  actual  facets  and  NPI  and  SP2  are  the  pseudo  facets.  A  graphical  illustration 
of  the  above  relation  is  shown  in  Figure  8b.  If  the  recovery  algorithm  is  applied  to  the  pseudo 
facets,  the  result  of  the  k!h  iteration  may  be  expressed  as: 

N/+J  =  N„  -  pkKk  NP2  (27) 

N2*+i  =  NP2  -  pkKk  NPi 

where  pk  and  Kk  are  computed  using  the  intermediate  facet  estimates  N/  and  N2*.  Let  us  focus 
our  attention  on  one  of  the  two  facets,  namely,  N/.  Since  Np;  and  NP2  are  constant,  new  estimates 
of  N;  result  solely  from  changes  in  the  factor  pkKk.  Since  pkKk  is  a  scalar,  the  facet  estimates  N/ 
must  lie  on  the  line  passing  through  the  vector  C  (Figure  8b).  This  line  constraint  implies  that  the 
convergence  of  N/  may  be  studied  by  analyzing  the  convergence  of  pkKk. 

We  assume  that  the  reflectance  estimates  pk  do  not  vary  substantially  from  the  actual 
reflectance  p.  This  assumption  is  based  on  the  observation  that  the  pseudo  reflectance  results 
from  the  multiple  reflections  of  light  rays  between  the  two  facets.  This  process  produces  a  pseudo 
reflectance,  p°,  that  maybe  expressed  as  an  infinite  exponential  series  in  the  actual  reflectance  value 
p.  Since  the  actual  albedo  must  be  less  than  unity,  the  higher  order  teims  in  the  series  may  be 
neglected  and  the  pseudo  albedo  is  governed  by  the  first  few  terms.  The  first  term  in  the  series  is 
in  fact  the  actual  albedo.  Therefore,  for  actual  albedo  values  that  are  not  close  to  unity  (say  p  < 
0.75),  the  pseudo  albedo  may  be  assumed  to  be  close  to  the  actual  albedo.  Hence,  we  make  the 
assumption  that  the  pseudo  albedo  and  all  intermediate  estimates  of  albedo  in  the  recovery  process 
do  not  vary  substantially  from  the  actual  albedo  value,  i.e.  pk  a  p.  Therefore,  variations  in  the 
factor  pkKk  are  dominated  primarily  by  variation  in  Kk. 

From  the  geometry  shown  in  Figure  8a,  we  see  that  the  orientation  of  the  two  facets  may  be 
determined  by  the  tilt  angle  6n.  The  interreflection  process,  in  a  sense,  tends  to  make  the  orientation 
of  each  facet  more  like  that  of  the  other  facet.  In  other  words,  as  shown  in  Figure  8b,  the  pseudo 
facets  are  guaranteed  to  have  a  smaller  tilt  angle  than  the  actual  facets.  Further,  the  interreflection 
kernel  K  is  a  monotonic  function  of  the  tilt  angle  6n.  This  is  shown  in  Figure  9,  where  K  is  plotted 
as  a  function  of  6n  for  different  values  of  the  facet  separation  distance  r.  The  first  estimate  of  the 
kernel,  namely,  K°,  is  less  than  the  actual  kernel  K  but  yet  greater  than  zero.  Equivalently,  the  facet 
estimate  N/  has  a  greater  tilt  angle  than  the  previous  estimate  but  less  than  that  of  the  actual  facet. 
With  each  iteration,  therefore,  the  kernel  estimates  increase  in  value  and  approach  the  actual  kernel 
value,  i.e.  K°  <  Kk  <  Kk+I  <  K.  Consequently,  the  facet  estimates,  N/,  start  from  NP1  and  move 
along  the  vector  C  to  finally  converge  at  Ny. 
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The  above  argument  holds  for  all  facets  whose  pseudo  albedo  values  do  not  vary  substan¬ 
tially  over  the  vector  C.  Figure  10a  shows  a  convergence  map  for  the  two-facet  case.  Each  point 
on  the  map  corresponds  to  an  instance  of  the  actual  facet  vector  N;;  the  tilt  angle  of  the  facets  vary 
along  the  concentric  circles  and  the  albedo  of  the  facets  vary  radially.  For  all  instances  of  N;,  the 
two  facets  are  assumed  to  be  separated  by  a  constant  distance  of  r  =  1.  For  each  instance  of  Ny,  the 
corresponding  pseudo-facet  Np;  is  computed  using  the  forward  interreflection  solution  and  plotted 
on  the  convergence  map  for  NPJ  shown  in  Figure  10b.  The  recovery  algorithm  was  independently 
applied  to  each  pseudo  facet  and  a  facet  tilt  error  was  computed  at  the  end  of  100  iterations.  The 
small  dots  in  Figure  10a  correspond  to  those  actual  facets  for  which  the  algorithm  successfully 
recovered  the  actual  facet  from  the  corresponding  pseudo  facet.  The  large  dots  correspond  to  the 
facets  for  which  the  algorithm  does  not  converge  at  the  actual  facet  but  rather  at  some  other  point. 
Similarly,  the  small  dots  and  large  dots  in  Figure  10b  correspond  to  the  pseudo  facets  that  do  and 
do  not  produce  accurate  actual  facet  estimates,  respectively. 

Note  that  the  algorithm  fails  to  produce  accurate  estimates  for  points  that  have  large  albedo 
values  (p  >  0.751  and  large  tilt  angles  (8n  >  70  degrees).  For  points  in  this  range  (large  dots), 
the  algorithm  does  in  fact  converge,  but  it  converges  to  other  facet  vectors  that  lie  in  between  the 
pseudo  facet  and  the  actual  facet  on  the  vector  C  (Figure  8b).  This  results  from  the  fact  that  facets 
with  two  different  tilt  angles  and  two  different  albedo  values  can  produce  the  same  pseudo  facets! 
This  is  also  illustrated  in  the  convergence  map  for  the  pseudo  facets  (Figure  10b)  where  constant 
albedo  curves  intersect  one  another,  thus  implying  that  two  different  actual  facets  can  result  in 
the  same  pseudo  facet.  Note  that  each  facet  separation  distance,  r,  results  in  a  different  set  of 
convergence  maps. 


K 


Figure  9:  The  interreflection  kernel  K  for  the  two-facet  case,  plotted  as  a  function  of  the 
facet  tilt  angle  0n ,  for  different  values  of  the  separation  distance  r. 


16 


Figure  10:  (a)  Convergence  map  for  actual  facets  N/.  r  =  1.  (b)  Convergence  map  for 
pseudo  facets  NP/.  r  =  1. 
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5.2  General  Case 


Conve.  "i.  for  the  general  case  of  surfaces  of  arbitrary  shape  (more  than  two  facets)  and  with 
varying  reflectance  is  very  difficult  to  prove.  This  is  because  the  algorithm  iterates  on  a  system 
of  equations  that  are  non-linear  functions  of  the  shape  and  reflectance  of  the  surface.  Hence,  the 
most  convincing  evidence  of  the  stability  and  accuracy  of  the  algorithm  lies  in  the  the  simulation 
results  of  the  previous  section  and  the  experimental  results  to  follow.  However,  fairly  convincing 
qualitative  statements  may  be  made  regarding  the  general  behavior  of  the  algorithm.  The  algorithm 
uses  the  pseudo  shape  and  reflectance  as  an  initial  estimate  of  the  actual  shape  and  reflectance. 
Since  the  pseudo  shape  is  less  concave  than,  but  yet  resembles  in  shape  to,  the  actual  shape,  the 
interreflections  produced  by  the  pseudo  shape  are  less  than  in  the  case  of  the  actual  surface  but 
yet  greater  than  zero.  Mod  Hng  of  these  non-zero  interreflections  is  expected  to  produce  a  better 
estimate  of  shape  and  ~“fiectance  in  the  next  iteration.  As  the  shape  and  reflectance  estimates 
approach  the  actual  one..,  the  eudo  shape  and  reflectance  are  almost  perfectly  accounted  for 
and  the  algorithm  converges.  Just  as  in  the  two-facet  case,  two  different  surface  shapes  with  two 
different  albedo  functions  can  produce  the  same  pseudo  shape.  However,  we  are  using  both  the 
pseudo  shape  as  well  as  the  pseudo  reflectance  together  to  estimate  actual  shape  and  reflectance. 
Hence,  the  probability  of  the  algorithm  converging  at  intermediate  shape  estimates  is  small  though 
not  zero.  In  general,  however,  lower  albedo  values  produce  lesser  interTeflections  and  therefore 
higher  likelihood  of  convergence.  Further,  in  all  our  simulations  and  experiments  we  confine 
ourselves  to  facets  whose  tilt  angle  are  less  than  70  degrees.  This  is  because  larger  tilt  angles 
produce  larger  facet  areas  that  result  in  poorer  approximations  to  the  interreflection  kernel.  In 
some  cases,  such  poor  approximations  can  lead  to  instability;  the  algorithm  may  produce  grossly 
erroneous  intermediate  estimates  that  finally  lead  to  divergence,  rather  than  convergence,  of  shape 
and  reflectance. 


6  Experimental  Results 

We  have  conducted  experiments  to  demonstrate  the  accuracy  and  practical  feasibility  of  the  shape 
and  reflectance  recovery  algorithm.  The  algorithm  was  applied  to  general  three-dimensional 
surfaces  as  well  as  surfaces  that  have  translational  symmetry.  A  512x480  CCD  camera  was 
used  to  view  the  surfaces.  Three  incandescent  lamps  were  used  to  illuminate  the  surfaces  from 
three  different  directions  to  extract  surface  orientation3  and  reflectance  information  by  photometric 
stereo.  The  brightness  of  each  source  was  determined  by  a  calibration  procedure  that  uses  flat 
surfaces  of  known  albedo  values. 

3 Surface  shape  is  obtained  by  integrating  surface  orientation. 
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6.1  Translational  Symmetry  Case 


Figure  1 1  and  Figure  12  show  the  results  for  objects  with  translational  symmetry  in  a  single 
direction.  Each  object  was  painted  with  dull  white  paint  to  give  it  a  matte  (Lambertian -like) 
reflectance.  In  each  case,  a  photo  of  the  object  is  shown  and  the  horizontal  line  in  the  photo 
represents  the  surface  points  that  were  used  by  the  recovery  algorithm.  The  cross-sectional  shape 
(actual  shape)  of  the  surface  was  determined  from  the  known  shape  of  the  object.  Due  to  the 
two-dimensional  nature  of  the  problem,  only  two  light  source  directions  were  needed  to  extract 
pseudo  shape  and  reflectance  estimates  by  photometric  stereo.  The  extracted  pseudo  albedo  value 
of  each  facet  is  represented  by  a  circle  in  the  reflectance  graph.  The  discrete  two-dimensional 
kernel  for  the  translational  symmetry  case  (Appendix  A.2)  was  used  by  the  recovery  algorithm  to 
ol.tain  the  actual  shape  and  reflectance  from  the  pseudo  ones.  The  intermediate  shape  estimates  are 
numbered  according  to  the  iteration  that  produced  them.  For  all  surfaces  in  Figures  1 1  and  12,  the 
shape  estimates  converge  to  reasonably  accurate  estimates  within  7  iterations  of  the  algorithm.  For 
each  surface,  the  mean  orientation  error  9e  (section  4.2)  was  computed  after  25  iterations  and  was 
found  to  be  less  than  2.5  degrees.  Note  that  the  albedo  estimates  converge  simultaneously  with  the 
orientation  estimates,  and  are  represented  by  the  small  dots  in  the  reflectance  graphs. 

Figures  12b  shows  a  convex  surface.  Note  that  for  a  convex  surface  the  pseudo  shape  and 
reflectance  estimates  are  equal  to  the  actual  ones.  Since  no  two  facets  on  this  surface  are  visible  to 
one  another  ( View  =  0),  the  algorithm  converges  at  the  pseudo  shape  and  reflectance  estimates. 


6.2  General  Case 

Figures  13  shows  a  photo  of  an  inverted  pyramid.  Again,  the  surface  is  painted  and  has  a  matte 
finish  .  In  this  case,  three  light  source  directions  were  used  to  extract  pseudo  shape  and  reflectance 
estimates  and  the  general  form  of  the  discrete  kernel  (equation  17)  was  used  by  the  recovery 
algorithm  to  extract  the  actual  shape  and  reflectance.  Figures  14a  and  14f  illustrate  isometric  and 
front  views  of  the  structure  of  the  inverted  pyramid  in  Figure  13.  Figures  14b  and  14g  show  the 
isometric  and  front  views  of  the  pseudo  shape  of  the  inverted  pyramid  extracted  by  photometric 
stereo.  The  pseudo  shapes  are  followed  by  a  few  intermediate  estimates  of  the  shape  produced  by 
the  recovery  algorithm.  The  convergence  graph  for  the  inverted  pyramid  is  shown  in  Figure  15. 
The  shape  estimate  converges  in  about  6  iterations  with  a  mean  orientation  error  Wt  ss  3  degrees. 


6.3  Discussion 

From  the  above  experiments  we  see  that  the  recovery  algorithm  performs  in  a  stable  manner  for 
a  variety  of  surface  shapes.  All  the  surfaces  used  in  the  experiments  have  high  albedo  values 
(approximately  0.9)  and  thus  exhibit  strong  interreflections.  Though  surface  albedo  was  not  known 
a-priori,  the  algorithm  was  successful  in  extracting  fairly  accurate  estimates  of  shape  and  reflectance 
from  the  pseudo  estimates.  Errors  in  the  recovered  shape  and  reflectance  estimates  are  caused  by 
the  following  factors: 
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•  The  Lambertian  assumption.  Though  all  surfaces  used  in  the  experiments  have  a  matte 
finish,  they  are  not  perfectly  Lambertian  in  reflectance.  Since  the  interreflection  model  used 
by  the  recovery  algorithm  is  based  on  the  Lambertian  assumption,  errors  in  the  shape  and 
reflectance  estimates  are  expected. 

•  In  the  above  results,  we  find  that  errors  in  recovered  shape  are  maximum  in  the  areas  where 
the  surface  is  discontinuous  in  orientation  (e.g.  roof  edges).  Facets  close  to  the  edge  that  are 
visible  to  one  another  are  expected  to  have  large  kernel  values.  However,  due  to  the  close 
proximity  of  these  facets,  the  discrete  kernel  does  not  provide  a  good  approximation  to  the 
actual  kernel  value.  As  a  result,  recovered  facet  orientations  tend  to  be  erroneous  in  these 
regions. 

In  our  experiments,  we  have  confined  ourselves  to  surface  orientations  with  tilt  angles  that  are 
less  that  70  degrees  with  respect  to  the  viewing  direction.  As  we  pointed  out  earlier,  large  tilt 
angles  would  result  in  large  facet  areas  for  which  the  kernel  given  by  equation  17  does  not  provide 
reasonable  approximations. 


7  Conclusion 

•  We  have  presented  in  this  paper  an  algorithm  for  recovering  the  shape  and  reflectance  of 
Lambertian  surfaces  in  the  presence  of  interreflections.  The  surfaces  may  be  of  arbitrary  but 
continuous  shape,  and  with  possibly  varying  and  unknown  reflectance. 

•  The  actual  shape  and  reflectance  are  recovered  from  the  pseudo  shape  and  pseudo  reflectance 
estimated  by  a  local  shape-from-intensity  method.  The  recovery  algorithm  may  therefore 
be  used  in  conjunction  with  any  shape-from-intensity  method  that  is  capable  of  producing 
accurate  pseudo  shape  and  pseudo  reflectance  estimates.  Thus,  the  algorithm  enhances  the 
performance,  and  hence  also  the  utility,  of  existing  shape-from-intensity  methods. 

•  We  have  analyzed  in  detail  the  convergence  properties  of  the  recovery  algorithm  for  the 
special  case  of  two  planar  facets.  Qualitative  arguments  that  support  convergence  for  the 
general  case  are  also  given. 

•  We  have  included  several  simulation  results  as  well  as  experimental  results  to  demonstrate 
the  robustness  and  accuracy  of  the  algorithm. 

In  this  paper,  we  have  restricted  ourselves  to  Lambertian  surfaces.  As  the  appearance  of 
a  Lambertian  surface  is  vantage  independent,  we  are  able  to  model  interreflections  in  a  compact 
form.  For  surfaces  with  other  reflectance  properties,  however,  surface  radiance  is  dependent  on  the 
viewing  direction.  In  such  cases,  both  the  forward  and  the  inverse  interreflection  problems  are  far 
more  difficult  to  solve.  As  future  research  we  are  interested  in  extending  our  analysis  to  specular 
surface,  and  subsequently,  to  surfaces  with  more  generic  reflectance  characteristics. 
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Figure  1 1 :  Experimental  results  for  surfaces  with  translational  symmetry  in  a  single  direction. 
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Figure  15:  Convergence  graph  for  the  in 


Inverted  Pyramid  (Figure  13) 


Isometric  View 


Photometric  Stereo  Result  (Pseudo  Shape) 


h  igure  14:  Shape  recovery  results  for  the  inverted  pyramid  shown  in  Figure  13. 


A  Appendix 

A.l  Radiometric  Definitions 


source 


Figure  16:  Basic  geometry  used  to  define  radiometric  terms. 


We  present  definitions  of  radiometric  terms  that  are  useful  in  the  analysis  of  interreflections. 
Detailed  derivations  and  descriptions  of  these  terms  are  given  by  Nicodemus  et  al.  [13].  Figure  16 
shows  a  surface  element  illuminated  by  a  source  of  light.  The  irradiance  E  of  the  surface  is  defined 
as  the  incident  flux  density  ( W/m~ 2): 


(28) 


where  d$t  is  the  flux  incident  on  the  area  dA  of  the  surface  element.  The  radiance  L  of  the  surface 
is  defined  as  the  flux  emitted  per  unit  fore-shortened  area  per  unit  solid  angle  (W/m-2.,sr-1).  The 
surface  radiance  in  the  direction  ( 6r ,  <pr)  is  determined  as: 

L  =  ** r 

dA  cosdr  dwr 


where  d2$r  is  the  flux  radiated  within  the  solid  angle  dwr.  The  Bi-Directional  Reflectance  Dis¬ 
tribution  Function  (BRDF)  of  a  surface  is  a  measure  of  how  bright  the  surface  appears  when 
viewed  from  a  given  direction,  when  it  is  illuminated  from  another  given  direction.  The  BRDF  is 
determined  as: 


(30) 
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A.2  Kernel  for  Single  Translational  Symmetry  Case 


Figure  17:  Cross-sectional  view  of  two  planar  facets 


that  are  infinite  in  the  x  direction. 


Forsyth  and  Zisserman  [3]  have  derived  the  discrete  interreflection  kernel  for  the  special  case  of  a 
three-dimensional  surface  that  has  translational  symmetry  in  a  single  direction.  Figure  17  shows 
a  cross-sectional  view  of  two  facets  that  are  infinite  in  the  x  direction.  The  kernel  /sT,y  is  derived 
[3]  by  integrating  along  the  x  direction  the  contribution  of  all  points  on  facet  j  to  the  radiance  of  a 
point  on  the  facet  i: 


Kij 


c  +  u  cosa 


(c2  +2cu‘cosa  +  w*2) 


1/2 


u'=b 


u  =a 


(31) 


where  o  is  the  angle  between  the  surface  normal  vectors  of  the  two  facets  and  the  parameter  u’ 
respresents  the  cross-sectional  length  of  the  facet  j.  Since  both  facets  are  infinite  in  length,  the 
same  kernel  value  is  valid  for  all  points  on  the  facet  i.  Therefore,  under  the  translation  symmetry 
assumption,  the  kernel  is  two-dimensional  in  that  it  need  only  be  evaluated  for  points  along  the 
cross-section  of  the  surface.  Note  that  the  above  kernel  is  valid  only  for  surfaces  that  are  infinite  in 
the  direction  of  symmetry.  However,  the  kernel  serves  as  a  good  approximation  [3]  for  points  that 
lie  around  the  middle  of  surfaces  that  are  long  though  finite  in  the  direction  of  symmetry. 
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